TabPFN: A Transformer That Solves Small Tabular Classification Problems in a Second
Dec 2, 2022
Speakers
Noah Hollmann
Speaker · 0 followers
Samuel Müller
Speaker · 0 followers
Katharina Eggensperger
Speaker · 0 followers
Frank Hutter
Speaker · 2 followers
About
We present TabPFN, a trained Transformer model that can do tabular supervised classification for small datasets in less than a second, needs no hyperparameter tuning and is competitive with state-of-the-art classification methods.TabPFN is entailed in the weights of our network, which accepts training and test samples as a set-valued input and yields predictions for the entire test set in a single forward pass. TabPFN is a Prior-Data Fitted Network (PFN) and is trained offline once, to approximate Bayesian inference on synthetic datasets drawn from our prior. Our prior incorporates ideas from causal learning: It entails a large space of structural causal models with a preference for simple structures. Afterwards, the trained TabPFN approximates Bayesian prediction on any unseen tabular dataset, without any hyperparameter tuning or gradient-based learning.On 30 datasets from the OpenML-CC18 suite, we show that our method outperforms boosted trees and performs on par with complex state-of-the-art AutoML systems with a 70× speedup. This increases to a 3 200× speedup when a GPU is available.We provide all our code and the trained TabPFN at https://anonymous.4open.science/r/TabPFN-2AEE. We also provide an online demo at https://huggingface.co/spaces/TabPFN/TabPFNPrediction.We present TabPFN, a trained Transformer model that can do tabular supervised classification for small datasets in less than a second, needs no hyperparameter tuning and is competitive with state-of-the-art classification methods.TabPFN is entailed in the weights of our network, which accepts training and test samples as a set-valued input and yields predictions for the entire test set in a single forward pass. TabPFN is a Prior-Data Fitted Network (PFN) and is trained offline once, to approxim…
Organizer
NeurIPS 2022
Account · 961 followers
Like the format? Trust SlidesLive to capture your next event!
Professional recording and live streaming, delivered globally.
Sharing
Recommended Videos
Presentations on similar topic, category or speaker
Look where you look! Saliency-guided Q-networks for generalization in visual Reinforcement Learning
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Hierarchical Abstraction for Combinatorial Generalization in Object Rearrangement
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Plan To Predict: Learning an Uncertainty-Foreseeing Model For Model-Based Reinforcement Learning
Watch later
Zifan Wu, …
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Perturbation Learning Based Anomaly Detection
Watch later
Jinyu Cai, …
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Why do we need SSL?
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
AnoShift: A Distribution Shift Benchmark for Unsupervised Anomaly Detection
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%